Methodological evaluation of bias in observational coronavirus disease 2019 studies on drug effectiveness.

BACKGROUND AND OBJECTIVE: Observational studies may provide valuable evidence on real-world causal effects of drug effectiveness in patients with coronavirus disease 2019 (COVID-19). As patients are usually observed from hospital admission to discharge and drug initiation starts during hospitalization, advanced statistical methods are needed to account for time-dependent drug exposure, confounding and competing events. Our objective is to evaluate the observational studies on the three common methodological pitfalls in time-to-event analyses: immortal time bias, confounding bias and competing risk bias. METHODS: We performed a systematic literature search on 23 October 2020, in the PubMed database to identify observational cohort studies that evaluated drug effectiveness in hospitalized patients with COVID-19. We included articles published in four journals: British Medical Journal, New England Journal of Medicine, Journal of the American Medical Association and The Lancet as well as their sub-journals. RESULTS: Overall, out of 255 articles screened, 11 observational cohort studies on treatment effectiveness with drug exposure-outcome associations were evaluated. All studies were susceptible to one or more types of bias in the primary study analysis. Eight studies had a time-dependent treatment. However, the hazard ratios were not adjusted for immortal time in the primary analysis. Even though confounders presented at baseline have been addressed in nine studies, time-varying confounding caused by time-varying treatment exposure and clinical variables was less recognized. Only one out of 11 studies addressed competing event bias by extending follow-up beyond patient discharge. CONCLUSIONS: In the observational cohort studies on drug effectiveness for treatment of COVID-19 published in four high-impact journals, the methodological biases were concerningly common. Appropriate statistical tools are essential to avoid misleading conclusions and to obtain a better understanding of potential treatment effects.

Statistical Analysis of Clinical COVID-19 Data: A Concise Overview of Lessons Learned, Common Errors and How to Avoid Them.

By definition, in-hospital patient data are restricted to the time between hospital admission and discharge (alive or dead). For hospitalised cases of COVID-19, a number of events during hospitalization are of interest regarding the influence of risk factors on the likelihood of experiencing these events. The same is true for predicting times from hospital admission of COVID-19 patients to intensive care or from start of ventilation (invasive or non-invasive) to extubation. This logical restriction of the data to the period of hospitalisation is associated with a substantial risk that inappropriate methods are used for analysis. Here, we briefly discuss the most common types of bias which can occur when analysing in-hospital COVID-19 data.